Communication over the Internet Using a 3d Agent with Real-time Facial Expression Analysis, Synthesis and Text to Speech Capabilities
نویسندگان
چکیده
We present a system for Internet communication that enhances traditional text-based chatting with real-time analysis and synthesis of the chat parties’ facial expressions. It is composed of three main modules: a real-time facial expression analysis component, a 3D agent with facial expression synthesis and text-to-speech capabilities – a talking head, and a communication module. So far we have realized a prototype to find out attractive ways of communication on the Internet and are currently experimenting on how to utilize this new type of modalities in chat communication. Keywords— facial expression, MPEG-4 FAP, 3D agent, chat
منابع مشابه
Real-time synthesis of Chinese visual speech and facial expressions using MPEG-4 FAP features in a three-dimensional avatar
This paper describes our initial work in developing a real-time audio-visual Chinese speech synthesizer with a 3D expressive avatar. The avatar model is parameterized according to the MPEG-4 facial animation standard [1]. This standard offers a compact set of facial animation parameters (FAPs) and feature points (FPs) to enable realization of 20 Chinese visemes and 7 facial expressions (i.e. 27...
متن کاملReal-time Synthesis of Chinese Visua using MPEG-4 FAP Features in a
This paper describes our initial work in developing a real-time audio-visual Chinese speech synthesizer with a 3D expressive avatar. The avatar model is parameterized according to the MPEG-4 facial animation standard [1]. This standard offers a compact set of facial animation parameters (FAPs) and feature points (FPs) to enable realization of 20 Chinese visemes and 7 facial expressions (i.e. 27...
متن کاملText2Video: Text-Driven Facial Animation using MPEG-4
We present a complete system for the automatic creation of talking head video sequences from text messages. Our system converts the text into MPEG-4 Facial Animation Parameters and synthetic voice. A user selected 3D character will perform lip movements synchronized to the speech data. The 3D models created from a single image vary from realistic people to cartoon characters. A voice selection ...
متن کاملA Mobile and Fog-based Computing Method to Execute Smart Device Applications in a Secure Environment
With the rapid growth of smart device and Internet of things applications, the volume of communication and data in networks have increased. Due to the network lag and massive demands, centralized and traditional cloud computing architecture are not accountable to the high users' demands and not proper for execution of delay-sensitive and real time applications. To resolve these challenges, we p...
متن کاملAcquisition of a 3D Audio-Visual Corpus of Affective Speech
Communication between humans deeply relies on our capability of experiencing, expressing, and recognizing feelings. For this reason, research on human-machine interaction needs to focus on the recognition and simulation of emotional states, prerequisite of which is the collection of affective corpora. Currently available datasets still represent a bottleneck because of the difficulties arising ...
متن کامل